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ABSTRACT 



Differences between comprehension of inductively 
presented material and of deductively presented materials as 
reflected by scores on multiple-choice and cloze tests of 
comprehension were sought using specifically constructed materials. 
The Logical Patterns Test was designed with a series of 
mult iple— choice tests in one section and a series of cloze tests in 
another. Each section has a deductive and inductive part. Subjects# 

33 college freshmen and sophomores# were given the test in two 
sessions. Data for the four subtests were analyzed using t-tests and 
correlation procedures. Significant differences were noted for 
presentation methods in both test types# the deductive passages being 
consistently more difficult. Examination of correlations among 
subtest scores revealed that multiple— choice tests appear to 
discriminate between comprehension of both presentation methods to a 
greater extent than do cloze tests. Tables and references are 
included, (MS) 
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The Adequacy of the Cloze In Measuring Comprehens ion 
of Different Logical patterns 

Ned H* Ratekin 
University of Northern Iowa 



This study explores the extent to which multiple-choice and cloze 
tests agree in their measurement of comprehension of material presented 
in two logical; patterns , induction and deduction* 

Considerable effort has been expended over the last few years in 
validating the cloze technique of testing reading comprehension* This 
validation for the most part has compared students 1 performance on cloze 
tests with their performance on standardized reading tests, or in some 
instances, on prepared multiple-choice tests. Despite the doubts expressed 
by Weaver and Kingston (1983), the high relationships found in such 
studies as Culhane (1970), Rankin (1970) and Bormuth (1967) have generally 
been accepted to Indicate that cloze tests approximate the general reading 



comprehension skill as measured In standardized tests 
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Weaver and Kingston* among others* hav^ claimed that eloze testing 
actually samples a different type of reading behavior than does multiple- 
choice testing, Bonnuth f s response (Bormuth* 1969) was to present evidence 
of a general reading ability that is represented equally by constructed 
multiple-choice tests and cloze tests over five sub-skills* He was imable 
to identify on the basis of student comprehension scores any factors or 
skills that were separate from this general factor on either the cloze or 
multiple^choice tests. 

This study is an attempt to look again at the assumption that cloze 
and multiple-choice tests equally reflect a general reading performance. 

Although predictions are high* cloze tests do not* of course* predict 
exact performance on other reading tests* This difference in variance may 
well represent a strength or advantage to cloze testing rather than indi- 
cation of lack of validity* It is on an intuitive level very tempting to 
view the cloze test* which requires close scrutiny of passages, as measur- 
ing some different processes* or skills than those measured by standard- 
ized multiple-choice tests. 

There is a long history of frustration and difficulty in isolating 
specific reading skills for analysis or for instruction in the skill. 
Mosberg's work (Mosberg* 1969) is an interesting example of attempts to 
categorize knowledge In this area. A variety of categories of factors have 
been suggested* such as; those related to organizational pattern* lin- 
guistic units related to comprehension, following patterns of thinking 
which result in adequate analysis of material, and the relationship of cog- 
nitive development to specific aspects of reading performance* These 
attempts to characterize skills in reading have identified factors in the 
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structure and organization of the materials read, or in the structure and 
organization of the mental processes of the reader as factors in the com- 
prehension process. 

There is a particular subset of these factors which refer simultan- 
eously to the structure of ideas in print and to the structure of particu- 
lar cognitive skills. For example, reading for chronological or sequential 
order is an application of seriation skills; making a generalization from 
cause and effect sequence is a type of conditional reasoning. The ability 
by the reader to identify and follow these reasoning patterns may underlie 
other skills of comprehension such as those labeled "interpretation, ” 
"evaluation," and "analysis 

A pair of general skills which fall in this category are the reasoning 
skills of induction and deduction related to patterns of idea development 
in written materials. The primary questions asked in this study center 
on the relative ability of cloze and multiple-choice tests to measure a 
reader's ability to comprehend materials in these two logical patterns. 

For this study, inductive reasoning is identified as the process of 
forming generalizations or drawing conclusions on the basis of facts, rea- 
sons or illustrations, by analogy, or through the use of comparison and 
contrast. An idea in print which presents a generalization by such evi- 
dence is identified as an inductive logical pattern. When this pattern 
appears in print, the generalization may either precede or follow the 
evidence. The logic in this type of reasoning is based on probability. 

Deductive reasoning in this study is defined as a process of drawing 
conclusions on the basis of logical necessity, or in following a form that 
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implies such a conclusion. The forms of concern are the traditional syllo- 
gistic forms , mathematical reasoning, transpositional reasoning and condi- 
tional reasoning. As in the inductive pattern, natural language passages 
identified as deductive do not necessarily follow the formal order of dis- 
covery, In a paragraph or longer passage, the conclusion may well precede 
the logical evidence. The premises on which the conclusion is based then 
appear as details of the passage. 

There is adequate evidence that these two patterns of reasoning exist 
and develop as unique abilities, Wetheriek (1969a and 1969b) found induc- 
tive inference to be a general capacity separate from any specific content, 
and that this capacity develops in two stages. In the first stage the per- 
son learns to Identify the subset of cues in common instances. In the 
second stage, the person learns to identify the subset of cues not present 
in other instances. Be found this last skill to be acquired last in child- 
hood and lost first in old age, implying to him that a short-term memory 
factor affects the second stage. 

According to Kofsky (1866) learning the induction process, in Plagetian 
terms, is learning resemblance sorting, consistency in sorting, and exhaust- 
ive classification, while the deductive process develops through learning 
multiple class memberships, horizontal classification and hierarchical re- 
classification, Roberge (197'*)) also identifies deductive reasoning with the 
mental operations of class reasoning and conditional reasoning. He has il- 
lustrated developmental improvement in deductive reasoning over grades 4-10. 

Although these patterns of reasoning are apparently different in form 
and development, there is little evidence to compare them in difficulty. 
Since class reasoning and conditional reasoning grow out of inductive ex- 
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perienee, it may be assumed that deduction begins to develop after some 
inductive skills have been established. 

It is assumed in this study that ability to follow these two logical 
patterns in written materials would be a skill related to successful read- 
ing comprehension* In fact, reading experts in their tests and other writ- 
ings frequently refer to the need to develop skill in discriminating logi- 
cal patterns, Jordan (1967), Karlin (1971) p* 229, Herber (1971). 

Despite the importance of this skill, standardized tests provide mini- 
mum opportunity to evaluate student abilities in a range of these patterns. 
Table I presents an analysis of the provision for inductive and deductive 
reasoning in six well-known standardized reading tests* The reading passa- 
ges in the comprehension portions of these tests have been designated as in- 
ductive or deductive. Narrative and descriptive material Is included in the 
inductive category* 



Table 1 

Per Cent of Inductive and Deductive passages 
and Related Questions in Six Standardized 
Reading Tests 



Reading Test 
Coop English, Form 1A 

Diagnostic Reading Tests Survey Form B 
Gates-McGinitie, Survey E. Form 1M 
Iowa Silent Feeding Test, Form Am 
Nelson Denny Reading Test, Form A 
Stanford Diagnostic Reading Test, Level 



Inductive Deductive 



passages 


Questions 


passages 


Questions 


100 


100 


0 


0 


83 


94 


17 


0 


76 


75 


24 


25 


100 


100 


0 


0 


100 


100 


0 


0 


68 


78 


32 


22 
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An extremely high incidence of inductive items was found in these 
tests. Part of the difference found between results of cloze and multiple- 
choice testing may be due to skills, such as deductive reasoning, which are 
not sampled on standardized tests. In such a case, the cloze test with 
the possibility for selecting a wider range of patterns, may give a more 
adequate measure of this ability for an individual reader. 

Hypotheses and Procedures 

In general the purpose of this study is to discover is any difference 
between comprehension of material presented inductively and comprehension 
of material presented deductively is equally reflected by scores on mul- 
tiple-choice and cloze tests. 

Specifically, the following two null hypotheses are tested in the study* 

1* There is no difference between comprehension scores of students 

reading materials identified as inductive and materials identified 
as deductive 

a. As measured by a multiple-choice test. 
b # As measured by a cloze test. 

2, There is no difference between correlations found when comparing 
inductive and deductive scores from multiple-choice tests and 
those scores from cloze tests. 

To test these hypotheses the test Comprehension of Logical patterns, 
was given to 33 college freshmen and sophomores who were among those en- 
rolled in a community college reading improvement program, students in the 
program were advised to enroll on the basis of their scoring below the 40th 
percentile on the Diagnostic Reading Test. 

The Logical Patterns Test, prepared especially for this study, is com- 
posed of two sections, one a series of multiple-choice tests and the other 
a series of cloze tests. Each section of the test is divided into two parts. 
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In one part the passages are presented inductively, and in the second part 
the passages follow a deductive pattern. These passages were selected from 
three different sources. Selections of approximately the same length were 
included in each part of the test from each of the three sources. The 
difficulty level of the passages was identified by the Pry scale as grade 10, 
grade 12 and college (difficult, near the end of the chart)* Multiple-choice 
questions were constructed for the first two parts of the test. Each of the 
40 items offered four foils, and required relating ideas in the passage. 

The cloze tests appeared with every 5th word deleted and required exact 
replacement for correct responses. Special answer sheets were prepared for 
both the multiple-choice and cloze items. The test was administered in two 
sessions. 

Figure I presents a design of the test which yields four subtest 
scores: MOT, MOD, Cloze-I, ClozeHD. 

Figure I 

Test I - Comprehension of Logical patterns 

Multiple«"Choice Cloze 

Inductive Deductive Inductive Deductive 



Sources 


12 3 


12 3 


12 3 


12 3 




I 


2 


3 


4 


Scores 


MO I 


MOD 


Cloze-I 


Cloze-D 




(20) 


(20) 


(139) 


(145) 



The data was analyzed hy t-tests of the significance or difference 
between correlated means, and by comparing correlations in the matrix 
formed from the four test scores* 
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Results and Discussion 

The first hypotheses requires a comparison of the difficulty students 
found in reading the two logical patterns. Table 2 presents the results of 
t-tes ts for the significance of difference between correlated means. These 
scores are expressed in per cent of correct responses. 

Table 2 

t-Tests of Difference Between Correlated Means on 
Multiple-Choice and Cloze Tests of Inductive and 
Deductive Passages 

Mean Per Ce nt Comprehens ion 

Inductive Deductive 





& 


X 


s 


X 




Multiple-Choice 


13.04 


51 


12.05 


45 


2,2175 


Cloze 


9.53 


30 


8.97 


23 


5.7423 



1 p < .025 

2 p <.001 

In relation to the accepted values of 75 per cent comprehension on 
multiple-choice and 40 per cent on cloze as levels of frustration, the pas- 
sages on this test must be viewed as quite difficult for the students. The 
Inductive-deductive difference on both the multiple-choice and the cloze 
tests are large enough to be significant. The deductive passages are con- 
sistently more difficult. 

Therefore, the hypothesis that there is no difference between compre- 
hension of inductive and deductive passages must be rejected on the basis of 
scores from both tests. The somewhat larger difference found on the cloze 
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test implies that it may be more sensitive to differences in the difficulty 
students found in understanding inductive and deductive passages. 

The second hypothesis requires a comparison of correlations among the 
subtest scores to determine if either test is more sensitive to individual 
differences among students in their ability to comprehend the two types of 
material. Table 3 presents the matrix of correlations found among the four 
test scores. 

Table 3 

Correlations Among Multiple-Choice and Cloze 
Subtests of Inductive and Deductive Comprehension 

Cloze-D 





2 


3 


4 


1. 


MC-1 . 24 


.40 


.20 


2, 


MC~D 


,55 


,38 


3, 


Cloze-I 




.80 



In general these correlations fall to reflect the degree of rela- 
tionship expected between tests of the same form or pattern* Beyond the 
possibility of low test reliability, these lower correlations may be due 
to the fact that each of the subtests differ from each other in at least 
one dimension, pattern or test form, and two of them differ in both dimen- 
sions. 

When test form is held constant, the cloze test correlates highest 
across the two patterns, with a correlation of *80, On the multiple-choice 
tests the correlation is .24. The two tests did not produce similar corre- 
lations between scores on inductive and scores on deductive materials. The 
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relatively high correlation of *80 shows a strong relationship between com- 
prehension of the two logical patterns when measured on the cloze test* 

But when measured by multiple choice items* the correlation .24* does not 
reach a level beyond chance* In spite of the necessary reservations due to 
test reliability and item validity* it seems that multiple-choice tests* 
prepared for that purpose, do discriminate between student f s comprehension 
of inductive and deductive patterns to a greater extent than do cloze tests. 

Therefore* the hypothesis that there is no difference between the cor- 
relations found when comparing inductive and deductive scores on the two 
tests must be rejected. 



Conclusions 

In this study the scores from a multiple-choice test prepared to measure 
comprehension of inductive and deductive logical patterns were compared to 
scores from cloze tests of the same patterns. Results on both tests indi- 
cated that the deductive patterns were significantly more difficult to 
understand than were the inductive passages. Also, the low correlation 
found on the multiple-choice test between comprehension of deductive end 
inductive patterns* compared to the high correlation of the two skills on 
the cloze tests* suggests that the multiple-choice tests prepared to measure 
comprehension of these patterns does a better job of discriminating dif- 
ferences in an individual read er*i ability in the two areas. 
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